Terminal device, and non-transitory computer readable medium storing program for terminal device

ABSTRACT

There is provided a terminal device. An image acquiring unit acquires a processing apparatus image serving as an image of a data processing apparatus. A display unit displays a request screen used for requesting a data processing apparatus to process data. At least one of processing apparatus images acquired by the image acquiring unit and at least one of data images indicating data that is requested to be processed are displayed on the request screen. In response to an operation of an operator of designating one of the processing apparatus images and one of the data images on the request screen, a request for processing the data indicated by the designated data image is transmitted to the data processing apparatus indicated by the designated processing apparatus image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/217,034 filed Jul. 22, 2016, which is based on and claims priorityunder 35 U.S.C. 119 from Japanese Patent Application No. 2016-049407filed with the Japan Patent Office on Mar. 14, 2016, the entire contentof which is hereby incorporated by reference.

TECHNICAL FIELD

The present invention relates to a terminal device, a data processingsystem, a non-transitory computer readable medium and a data processingmethod.

RELATED ART

When a terminal device transmits a data processing request to a dataprocessing apparatus, it is necessary to acquire an address of the dataprocessing apparatus on a network and establish a connection between theterminal device and the data processing apparatus. However, there arecases in which an operation of connecting the terminal device with thedata processing apparatus is complicated.

SUMMARY

According to an aspect of the embodiments of the present invention,there is provided a terminal device, comprising: an image acquiring unitthat acquires a processing apparatus image serving as an image of a dataprocessing apparatus; and a display unit that displays a request screenused for requesting a data processing apparatus to process data, whereinat least one of processing apparatus images acquired by the imageacquiring unit and at least one of data images indicating data that isrequested to be processed are displayed on the request screen, andwherein in response to an operation of an operator of designating one ofthe processing apparatus images and one of the data images on therequest screen, a request for processing the data indicated by thedesignated data image is transmitted to the data processing apparatusindicated by the designated processing apparatus image.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments of the present invention will be described indetailed based on the following figures, wherein:

FIG. 1 is a diagram illustrating a data processing system according to afirst exemplary embodiment of the present invention;

FIG. 2 is a block diagram illustrating a configuration of a terminaldevice included in the data processing system illustrated in FIG. 1;

FIG. 3 is a front view illustrating a smart phone as an example of theterminal device illustrated in FIG. 2;

FIG. 4 is a block diagram illustrating a configuration of a wirelesscommunication apparatus included in the data processing systemillustrated in FIG. 1;

FIG. 5 is a block diagram illustrating a configuration of a managementapparatus included in the data processing system illustrated in FIG. 1;

FIG. 6 is a block diagram illustrating a configuration of an informationprocessing apparatus included in the data processing system illustratedin FIG. 1;

FIG. 7A is a diagram schematically illustrating an aspect ofphotographing a data processing apparatus through a terminal device inorder to acquire a processing apparatus image, and FIG. 7B is a diagramillustrating a state in which the processing apparatus image of thephotographed data processing apparatus is displayed on a display unit ofthe terminal device;

FIG. 8A is a diagram illustrating a first example of a request screendisplayed on a display unit of a terminal device, and FIG. 8B is adiagram illustrating a second example of the request screen displayed onthe display unit of the terminal device;

FIG. 9A is a diagram for describing a first example of an operation ofrequesting a data processing apparatus to process data on a requestscreen, and FIG. 9B is a diagram for describing a second example of theoperation of requesting the data processing apparatus to process data onthe request screen;

FIGS. 10A and 10B are diagrams for describing a third example of theoperation of requesting the data processing apparatus to process data onthe request screen;

FIG. 11 is a diagram for describing an operation of requesting a dataprocessing apparatus to process data when plural processing apparatusimages are displayed on a request screen;

FIGS. 12A to 12D are diagrams illustrating examples of a data selectionscreen displayed on a display unit of a terminal device, and FIG. 12Eand FIG. 12F are diagrams illustrating states in which the dataselection screen illustrated in FIG. 12B is operated;

FIG. 13A is a diagram illustrating an example of a data display screendisplayed on a display unit of a terminal device, FIG. 13B is a diagramillustrating a third example of the request screen displayed on thedisplay unit of the terminal device, and FIG. 13C is a diagram fordescribing an operation of requesting a data processing apparatus toprocess data on the request screen illustrated in FIG. 13B;

FIG. 14 is a diagram for schematically describing an operation ofacquiring a processing apparatus image of a data processing apparatusthrough a terminal device;

FIGS. 15A and 15B are diagrams illustrating examples of a processingapparatus display screen displayed on a display unit of a terminaldevice, and FIG. 15C is a diagram illustrating a request screendisplayed on the display unit after FIG. 15B is displayed;

FIG. 16A is a diagram illustrating an example of a processing apparatusselection screen displayed on a display unit of a terminal device, andFIG. 16B is a diagram illustrating a request screen displayed on thedisplay unit after the processing apparatus selection screen illustratedin FIG. 16A is operated;

FIG. 17 is a diagram illustrating an example of a mode selection screendisplayed on a display unit of a terminal device;

FIG. 18 is a flowchart for describing an operation when a mode of theterminal device illustrated in FIG. 1 is selected;

FIG. 19 is a flowchart for describing an operation in a camera selectionmode of the terminal device illustrated in FIG. 1;

FIG. 20 is a flowchart for describing an operation in a sketch mapselection mode of the terminal device illustrated in FIG. 1;

FIG. 21 is a flowchart for describing an operation in a data selectionmode of the terminal device illustrated in FIG. 1;

FIG. 22 is a perspective view illustrating a wearable computer used as aterminal device in a data processing system according to a secondexemplary embodiment of the present invention;

FIG. 23 is a diagram illustrating an example of a request screendisplayed on a display unit of the wearable computer illustrated in FIG.22; and

FIG. 24 is a diagram illustrating a data processing system according toa third exemplary embodiment of the present invention.

DETAILED DESCRIPTION First Exemplary Embodiment

Hereinafter, embodiments for carrying out the present invention will bedescribed with reference to the appended drawings. FIG. 1 illustrates adata processing system 10 according to a first exemplary embodiment ofthe present invention. The data processing system 10 includes a terminaldevice 100, a wireless communication apparatus 300, for example, threedata processing apparatuses 400A, 400B, and 400C, and a managementapparatus 500 which are connected to a network 600.

The data processing apparatus 400A, the data processing apparatus 400B,and the data processing apparatus 400C are hereinafter referred tocollectively as a “data processing apparatus 400” when they need not beparticularly distinguished from one another. The following descriptionwill proceed with an example in which the data processing system 10includes the three data processing apparatuses 400, but the dataprocessing system 10 preferably includes at least one data processingapparatus 400.

The terminal device 100 is preferably a portable apparatus, and forexample, a smart phone, a tablet apparatus, a wearable computer, or thelike may be used. The terminal device 100 is preferably connected to thenetwork 600 through the wireless communication apparatus 300 in awireless manner but may be connected to the network 600 in a wiredmanner. The following description will proceed with an example in whicha smart phone serving as a portable apparatus connectable to the network600 through the wireless communication apparatus 300 is used as theterminal device 100.

FIG. 2 is a block diagram illustrating a configuration of the terminaldevice 100. As illustrated in FIG. 2, the terminal device 100 includesan operating unit 110. The operating unit 110 is used as an inputapparatus that receives an instruction of an operator according to anoperation of the operator, and for example, a touch panel or the likemay be used.

The terminal device 100 further includes a global positioning system(GPS) receiving unit 120. The GPS receiving unit 120 receives radiowaves from GPS satellites 990.

The terminal device 100 further includes a wireless communication unit130. The wireless communication unit 130 connects the terminal device100 with the network 600 by performing communication of a wirelessnetwork such as Wi-Fi and performing wireless communication with thewireless communication apparatus 300.

The terminal device 100 further includes an acceleration sensor 140. Theacceleration sensor 140 detects a posture and an inclination of theterminal device 100 with respect to the ground surface. The accelerationsensor 140 further detects whether or not a vertical direction of theterminal device 100 with respect to the ground surface is right.

The terminal device 100 further includes an electronic compass 150. Theelectronic compass 150 detects a direction of the terminal device 100.For example, a geomagnetic sensor or the like may be used as theelectronic compass 150.

The terminal device 100 further includes a display unit 160. Forexample, a liquid crystal display (LCD) panel may be used as the displayunit 160.

The terminal device 100 further includes a camera apparatus 170. Thecamera apparatus 170 is used as both an image acquiring unit and aphotographing apparatus, and photographs a subject, for example, usingthe display unit 160 as a finder.

The terminal device 100 further includes a memory 180. The memory 180stores, for example, image data photographed by the camera apparatus170.

The terminal device 100 further includes a control unit 102. The controlunit 102 controls the operating unit 110, the GPS receiving unit 120,the wireless communication unit 130, the acceleration sensor 140, theelectronic compass 150, the display unit 160, the camera apparatus 170,and the memory 180 according to a program. The control unit 102 furtherperforms image processing.

The control unit 102 further calculates the position of the terminaldevice 100 on earth based on a radio wave that is transmitted from theGPS satellite 990 and received by the GPS receiving unit 120. Asdescribed above, the control unit 102 obtains latitude and longitudeposition coordinates of the terminal device 100 and acquires positioninformation of the terminal device 100.

The position information of the terminal device 100 may be acquired by amethod other than a method of acquiring GPS radio waves. For example,the position information of the terminal device 100 may be acquired byWi-Fi positioning using a wireless LAN. Thus, even when the terminaldevice 100 does not include the GPS receiving unit 120, it is possibleto acquire the position information of the terminal device 100 throughplural wireless LAN access points.

FIG. 3 is a diagram illustrating the terminal device 100 at a frontview. As illustrated in FIG. 3, the display unit 160 is arranged on thefront side of the terminal device 100 so that display content can beviewed from the front side of the terminal device 100. The operatingunit 110 is arranged on the front side of the terminal device 100,similarly to the display unit 160 so that an operation can be performedon the front side.

The terminal device 100 includes, as the camera apparatus 170, a cameraapparatus 170A that photographs an area in front of the terminal device100 and a camera apparatus 170B that photographs an area behind theterminal device 100. The camera apparatus 170A and the camera apparatus170B are referred to collectively as a “camera apparatus 170” unlessthey need not be particularly distinguished from each other.

The terminal device 100 further includes an auxiliary operating unit112. The auxiliary operating unit 112 assists an operation performed bythe operating unit 110 and is arranged, for example, on the front sideof the terminal device 100. For example, a push button type switchapparatus may be used as the auxiliary operating unit 112.

FIG. 4 is a block diagram illustrating a configuration of the wirelesscommunication apparatus 300. The wireless communication apparatus 300functions as a wireless communication access point, and includes awireless communication unit 310, a communication interface (I/F) unit320, and a control unit 302 as illustrated in FIG. 4. The wirelesscommunication unit 310 performs wireless communication with the terminaldevice 100. The communication I/F unit 320 connects the wirelesscommunication apparatus 300 to the network 600 so that communication canbe performed. The control unit 302 controls the wireless communicationunit 310 and the communication I/F unit 320.

FIG. 5 is a block diagram illustrating a configuration of the managementapparatus 500. The management apparatus 500 is an apparatus thatcentrally manages the data processing apparatuses 400 connected to thenetwork 600, and includes an operating unit 510 as illustrated in FIG.5. The operating unit 510 is used to operate the management apparatus500, and for example, a keyboard or the like may be used as theoperating unit 510.

The management apparatus 500 further includes a communication I/F unit520. The communication I/F unit 520 connects the management apparatus500 to the network 600 so that communication can be performed. Themanagement apparatus 500 further includes a storage unit 530, a displayunit 540 such as a monitor, a memory 550, and a control unit 502 thatcontrols the storage unit 530, the display unit 540, and the memory 550.

For example, a hard disk drive (HDD) may be used as the storage unit530. The storage unit 530 stores position information of each of thedata processing apparatuses 400, shape information of each of the dataprocessing apparatuses 400, a communication protocol that can be usedfor communication by each of the data processing apparatuses 400, andidentification information of each of the data processing apparatuses400 in a database thereof so that the information can be read by thecontrol unit 502. Here, examples of the position information of the dataprocessing apparatus 400 include latitude and longitude positioncoordinates that can be acquired by a GPS, position coordinates acquiredby Wi-Fi positioning, and position coordinates obtained by an indoormessaging system (IMES).

As the position information of the data processing apparatus 400, aresult measured when each information processing apparatus is installedis stored in the storage unit 530. The data processing apparatus 400 mayhave a function of measuring a position, and the position of the dataprocessing apparatus 400 measured by the data processing apparatus 400may be, for example, automatically, stored in the storage unit 530 viathe network 600.

As the identification information of the data processing apparatus 400,an address on the network 600 may be used, and an ID, an IP address, orthe like of the data processing apparatus 400 in the network 600 may befurther used.

FIG. 6 is a block diagram illustrating a configuration of the dataprocessing apparatus 400. The data processing apparatus 400 is anapparatus that processes data based on a request from the terminaldevice 100. The following description will proceed with an example inwhich an image forming apparatus serving as an apparatus performing aprocess of forming an image from data based on the request from theterminal device 100 is used as each of the data processing apparatuses400A, 400B, and 400C, but the present invention is not limited to theimage forming apparatus, and any apparatus that processes informationsuch as data based on the request from the terminal device 100 can beused as the information processing apparatus.

For example, a projector, a display, or the like serving as an apparatusthat processes data, which outputs image data, based on the request fromthe terminal device 100 may be used as the data processing apparatus400. For example, an original reading apparatus (scanner) serving as anapparatus that processes data, which reads an image and transmits readimage data, for example, to the terminal device 100 or the like, basedon the request from the terminal device 100 may be used as the dataprocessing apparatus 400. Further, for example, a personal computer thatcan perform various processes, for example, data transmitted from theterminal device 100 based on the request from the terminal device 100may be used as the data processing apparatus 400.

The following description will proceed with an example in which the dataprocessing apparatuses 400A, 400B, and 400C are image formingapparatuses serving as the same type of apparatuses that perform thesame process, but the plural data processing apparatuses 400 connectedto the network 600 may be different types of apparatuses that performdifferent processes on information. For example, the data processingapparatus 400A may be an image forming apparatus, the data processingapparatus 400B may be a projector, and the data processing apparatus400C may be a personal computer.

As illustrated in FIG. 6, the data processing apparatus 400 includes acommunication I/F unit 410, and the communication I/F unit 410 connectsthe data processing apparatus 400 to the network 600 so thatcommunication can be performed. The data processing apparatus 400further includes a storage unit 420. For example, a HDD may be used asthe storage unit 420.

The data processing apparatus 400 further includes a sheet feeding unit440, an image forming unit 430, and an ejection unit 450. The sheetfeeding unit 440 supplies a sheet to the image forming unit 430, theimage forming unit 430 forms an image on the supplied sheet, and theejection unit 450 ejects the sheet on which the image is formed to theoutside of the data processing apparatus 400.

The data processing apparatus 400 further includes a control unit 402.The control unit 402 controls the communication I/F unit 410, thestorage unit 420, the image forming unit 430, the sheet feeding unit440, and the ejection unit 450.

FIG. 7A is a diagram schematically illustrating a first example of anoperation of acquiring a processing apparatus image 4001 serving as animage of the data processing apparatus 400 through the terminal device100 and schematically illustrates a state in which the data processingapparatus 400 is photographed by the terminal device 100. As illustratedin FIG. 7A, in order to photograph the terminal device 100, for example,an operation of pointing a camera apparatus 170B arranged on an oppositeside to the operating unit 110 and the display unit 160 of the terminaldevice 100 at the data processing apparatus 400 and then pushing, forexample, the auxiliary operating unit 112 is performed. In FIG. 7A, aclock 922 hangs on a wall surface behind the side on which the dataprocessing apparatus 400 is installed.

FIG. 7B illustrates a photograph captured by photographing the dataprocessing apparatus 400 as illustrated in FIG. 7A. The photograph isdisplayed on the display unit 160, and the processing apparatus image4001 serving as the image of the data processing apparatus 400 and aclock image 9221 serving as an image of the clock are shown on thephotograph. As described above, in the terminal device 100, the cameraapparatus 170B is used as both the photographing apparatus and the imageacquiring unit that acquires the processing apparatus image 4001.

Hereinafter, a screen on which the processing apparatus image 4001 isdisplayed on the display unit 160 as illustrated in FIG. 7B is referredto as a “processing apparatus display screen 160A.” The processingapparatus image 4001 displayed on the processing apparatus displayscreen 160A is preferably an image capable of specifying one dataprocessing apparatus 400 and need not be necessarily an image obtainedby photographing the camera apparatus 170. More specifically, theprocessing apparatus image 4001 may be a photograph of the dataprocessing apparatus 400 that is photographed in advance and stored, forexample, in a database installed in the storage unit 530 (see FIG. 5).The processing apparatus image 4001 may be an icon or the like capableof specifying one data processing apparatus 400 instead of thephotograph.

FIG. 8A illustrates a first example of a request screen 160B serving asa screen that is displayed on the display unit 160 and used forrequesting the data processing apparatus 400 to process data. Asillustrated in FIG. 8A, the processing apparatus image 4001 and a dataimage 960D indicating data that is requested to be processed by the dataprocessing apparatus 400 are displayed on the request screen 160B. Inthe request screen 160B illustrated in FIG. 8A, the data image 960D isdisplayed on an upper left portion of the display unit 160 that is apredetermined position of the display unit 160.

Here, examples of data indicated by the data image 960D include a textfile, an image file, and a data file created by a specific applicationsuch as word processing software.

FIG. 8B illustrates a second example of the request screen 160B. In thefirst example of the request screen 160B illustrated in FIG. 8A, thedata image 960D is displayed at a predetermined position of the displayunit 160. On the other hand, in the second example, the data image 960Dis displayed such that at least one data image 960D is arranged at aposition overlapping the processing apparatus image 4001. In the secondexample, the plural data images 960D are arranged and displayed to havea depth in a direction crossing the surface of the display unit 160.

In the examples of FIGS. 8A and 8B, one processing apparatus image 4001and the plural data images 960D are displayed on the request screen160B, but it is desirable that at least one processing apparatus image4001 and at least one data image 960D be displayed on the request screen160B.

FIG. 9A illustrates a first example of an operation of requesting thedata processing apparatus 400 to process data on the request screen160B. As illustrated in FIG. 9A, in the first example, in order torequest the data processing apparatus 400 to process data, the operatordrags the data image 960D indicating data that is desired to beprocessed on the operating unit 110 serving as a touch panel using afinger, and drags the dragged data image 960D onto the processingapparatus image 4001 indicating the data processing apparatus 400 thatis desired to process the dragged data image 960D.

FIG. 9B illustrates a second example of the operation of requesting thedata processing apparatus 400 to process data on the request screen160B. In the second example, the operator requests the data processingapparatus 400 to process data by making a gesture. More specifically,the operator selects the data image 960D by making a gesture of pullingthe data image 960D in space, and requests the data processing apparatus400 to process data indicated by the selected data image 960D by makinga gesture of turning a finger in space in the state in which the dataimage 960D is selected.

In order to perform the operation as in the second example describedabove with reference to FIG. 9B, for example, it is necessary for theoperator to wear a ring type or a wristwatch type of wearable terminal810. In addition to the configuration illustrated in FIG. 2, theterminal device 100 preferably includes a position detecting unit (notillustrated) that detects the position of the wearable terminal 810 andinputs a detection result to the control unit 102. Instead of installingthe position detecting unit in the terminal device 100, the terminaldevice 100 may be equipped with a position detecting apparatus 820 thatdetects the position of the wearable terminal 810.

FIGS. 10A and 10B illustrate a third example of the operation ofrequesting the data processing apparatus 400 to process data on therequest screen 160B. In the third example, similarly to the secondexample, the operator requests the data processing apparatus 400 toprocess data through a gesture of the operator. More specifically, theoperator selects the data image 960D by making a gesture of pointing thedata image 960D indicating data that is desired to be processed in spaceand further making a gesture of grabbing the data image 960D asillustrated in FIG. 10A.

Then, the data image 960D in the grabbed state is moved to overlap theprocessing apparatus image 4001 by moving the hand in space in a stateof grabbing the data image 960D, as illustrated in FIG. 10B. Then, thedata processing apparatus 400 is requested to process the data indicatedby the data image 960D by making a gesture of opening fingers in thestate in which the data image 960D overlaps the processing apparatusimage 4001.

In order to perform the operation as in the third example describedabove with reference to FIGS. 10A and 10B, the hand of the operator isphotographed through the camera apparatus 170A or the camera apparatus170B, photographed data is input to the control unit 102, the positionof the hand of the operator is calculated through the control unit 102,and the control unit 102 causes an image of the hand to be displayed onthe request screen 160B based on the calculated position of the hand.

In the above-described third example, the camera apparatus 170B is usedto photograph the hand of the operator. Thus, the operator need not wearthe wearable terminal 810, and the position detecting apparatus 820 neednot be installed in the terminal device 100 as in the second example.

FIG. 11 illustrates a third example of the request screen 160B. In thefirst and second examples of the request screen 160B, one processingapparatus image 4001 is displayed on the request screen 160B (see FIGS.8A and 8B). On the other hand, in the third example, two processingapparatus images 4001 are displayed on the request screen 160B. In therequest screen 160B, the data processing apparatus 400 positioned to becloser to the terminal device 100 among the data processing apparatuses400 indicated by the plural processing apparatus images 4001 to bedisplayed is displayed with a larger size on the request screen 160B.

In the third example of the request screen 160B illustrated in FIG. 11,in order to request the data processing apparatus 400 to process data,it is desirable to move the data image 960D indicating the data that isdesired to be processed to the processing apparatus image 4001indicating the data processing apparatus 400 that is desired to processon the request screen 160B. In order to move the data image 960D, theoperator may drag and drops the data image 960D on the operating unit110 configured with the touch panel, the operator may make a gesture inspace in the state in which the wearable terminal 810 is worn, or theoperator may make a gesture in space so that, for example, motion of thehand can be detected by the camera apparatus 170B.

Each of FIGS. 12A to 12D illustrates a data designation screen 160Cserving as a screen that is displayed on the display unit 160 and usedfor designating data that is requested to be processed by the dataprocessing apparatus 400. In the data designation screen 160C, at leastone data image 960D is displayed, and when there are plural data images960D to be displayed, the data images 960D are displayed to have a depthin a direction crossing the data designation screen 160C.

In other words, in the example illustrated in FIG. 12A, the plural dataimages 960D are displayed on the data designation screen 160C such thatthe plural data images 960D are arranged on one plane oblique to thedata designation screen 160C. In the example illustrated in FIG. 12B,the plural data images 960D are displayed on the data designation screen160C such that the plural data images 960D are arranged on a sphericalsurface. In the example illustrated in FIG. 12C, the plural data images960D are displayed on the data designation screen 160C such that theplural data images 960D are arranged on one curved band. In the exampleillustrated in FIG. 12D, the plural data images 960D are displayed suchthat the plural data images 960D are arranged on plural planes that arearranged to be oblique to the data designation screen 160C and parallelto each other.

FIGS. 12E and 12F illustrate the data designation screens 160C after theoperation is performed in the state illustrated in FIG. 12B. Morespecifically, FIG. 12E illustrates the plural data images 960D as if theplural data images 960D are viewed in such manner that a point of viewis moved to a position close to the spherical surface formed by theplural data images 960D arranged as illustrated in FIG. 12B. FIG. 12Fillustrates the plural data images 960D as if the plural data images960D are viewed from a point of view that is moved to the inside of thespherical surface formed by the plural data images 960D.

As described above, by performing the operation of moving the point ofview from which the plural data images 960D displayed to have the depthare viewed, it is possible to more easily find a desired data image 960Dand more easily designate a desired data image 960D.

The data designation screens 160C illustrated in FIG. 12A to FIG. 12Fmay be operated, for example, by operating the touch panel by theoperator, making a gesture by the operator in the state in which thewearable terminal 810 (see FIGS. 9A and 9B) is worn, or making a gestureby the operator while detecting the position of the finger of theoperator through the camera apparatus 170B. More specifically, forexample, a pointed data image 960D can be selected by making a gestureof pointing a specific data image 960D in space, and data indicated bythe selected data image 960D can be designated as data to be processedby making the gesture of turning the fingertip in the state in which thedata image 960D is selected.

FIG. 13A illustrates an example of a data display screen 160F serving asa screen that is displayed on the display unit 160 and used fordisplaying the data image 960D. In the example of FIG. 13A, for example,the data image 960D selected through the data designation screens 160Cillustrated in FIGS. 12A to 12D is displayed on the data display screen160F. In this example, for example, “please select apparatus throughcamera” is displayed on the data display screen 160F as a display 160 kfor encouraging an operation of selecting the data processing apparatus400 that is requested to process data.

FIG. 13B is a diagram illustrating a fourth example of the requestscreen 160B displayed on the display unit of the terminal device 100 asan example of a screen displayed after due to the encouragement by thedisplay 160 k, for example, the data processing apparatus 400 isphotographed through the camera apparatus 170, and the data processingapparatus 400 is selected. In the first and second examples of therequest screen 160B, the plural data images 960D are displayed on therequest screen 160B (see FIGS. 8A and 8B), and one data image 960Dindicating data that is requested to be processed among the plural dataimages 960D is designated on the request screen 160B. On the other hand,in the fourth example, one data image 960D selected on the datadesignation screen 160C is displayed on the request screen 160B.

In the fourth example, in order to request the data processing apparatus400 to process data, as illustrated in FIG. 13C, for example, theoperator performs an operation of dragging the data image 960Dindicating the data that is desired to be processed on the operatingunit 110 serving as the touch panel using the finger and then draggingthe dragged data image 960D onto the processing apparatus image 4001indicating the data processing apparatus 400 that is desired to process.

FIG. 14 is a diagram schematically illustrating a second example of theoperation of acquiring the processing apparatus image 4001 serving asthe image of the data processing apparatus 400 through the terminaldevice 100. In the first example of the operation of acquiring theprocessing apparatus image 4001, in order to acquire the processingapparatus image 4001, the data processing apparatus 400 is photographed,for example, using the camera apparatus 170B (FIG. 7A). On the otherhand, in the second example, in order to acquire the processingapparatus image 4001, for example, the camera apparatus 170B is pointedat a specific direction, and the camera apparatus 170B is operated, forexample, using the operating unit 110. As a result, when the cameraapparatus 170B is operated, the image of the data processing apparatus400 that is arranged in a direction in which the data processingapparatus 400 can be photographed through the camera apparatus 170B isacquired.

In the example illustrated in FIG. 14, the data processing apparatus 400is positioned at the opposite side to the camera apparatus 170B withrespect to an obstacle 924 such as a cabinet and thus unable tophotograph the data processing apparatus 400 through the cameraapparatus 170B. For this reason, it is not possible to acquire theprocessing apparatus image 4001 by photographing the data processingapparatus 400 through the camera apparatus 170B using the first example.However, in the second example, even when the data processing apparatus400 is arranged at the position at which it is not possible tophotograph the data processing apparatus 400 through the cameraapparatus 170B, it is possible to acquire the processing apparatus image4001 indicating the data processing apparatus 400 positioned in thedirection in which the data processing apparatus 400 can be photographedthrough the camera apparatus 170B.

FIG. 15A illustrates a second example of the processing apparatusdisplay screen 160A that is displayed on the display unit 160 and servesas a screen on which the processing apparatus image 4001 acquired by theoperation described with reference to FIG. 14 is displayed. In the firstexample of the processing apparatus display screen 160A, the photographof the data processing apparatus 400 photographed through the cameraapparatus 170B is used as the processing apparatus image 4001 (see FIG.7A). On the other hand, in the second example, the photograph of thedata processing apparatus 400 that is photographed in advance andstored, for example, in the storage unit 530 (see FIG. 5) or an iconindicating the data processing apparatus 400 is used as the processingapparatus image 4001.

In the second example, in order to facilitate understanding of apositional relation between a position and a direction of the cameraapparatus 170B and the data processing apparatus 400 indicated by theprocessing apparatus image 4001, the processing apparatus image 4001 isdisplayed to overlap the photograph captured by the camera apparatus170B when the camera apparatus 170B is operated. For example, in theexample illustrated in FIG. 15A, the processing apparatus image 4001 isdisplayed to overlap an obstacle image 9241 serving as an image of theobstacle 924 to indicate that the data processing apparatus 400 ispositioned at a hidden position behind the obstacle 924.

FIG. 15B illustrates a screen displayed after the second example of theprocessing apparatus display screen 160A illustrated in FIG. 15A isdisplayed as a third example of the processing apparatus display screen160A displayed on the display unit 160. In the second example of theprocessing apparatus display screen 160A, the processing apparatus image4001 is displayed on the processing apparatus display screen 160A sothat the positional relation between the obstacle 924 or the like andthe data processing apparatus 400 can be easily understood. On the otherhand, in the third example of the processing apparatus display screen160A, the processing apparatus image 4001 is displayed at apredetermined position of the processing apparatus display screen 160A,for example, on a lower right portion of the processing apparatusdisplay screen 160A.

Transition from the processing apparatus display screen 160A illustratedin FIG. 15A to the processing apparatus display screen 160A illustratedin FIG. 15B may be performed, for example, after a predetermined periodof time elapses or may be performed, for example, by an operation of theoperator of tapping the processing apparatus image 4001 on theprocessing apparatus display screen 160A.

FIG. 15C illustrates a fifth example of the request screen 160Bdisplayed on the display unit 160. In the fifth example of the requestscreen, for example, the plural data images 960D are displayed inaddition to the display of the third example of the processing apparatusdisplay screen 160A illustrated in FIG. 15B. In order to request thedata processing apparatus 400 to process data on this screen, theoperator preferably performs, for example, the operation of dragging thedata image 960D indicating the data that is desired to be processed andthen dragging the data image 960D onto the processing apparatus image4001 indicating the data processing apparatus 400 that is desired toprocess.

FIG. 16A illustrates a processing apparatus designation screen 160Dserving as a screen that is displayed on the display unit 160 and usedfor designating the data processing apparatus 400 that is requested toprocess. In the processing apparatus designation screen 160D, a sketchmap of an area around the position of the terminal device 100 isdisplayed, the position of the terminal device 100 on the sketch map isfurther indicated, for example, by a mark 1001 such as an icon, and thedata processing apparatuses 400 installed within the range displayed onthe sketch map and the positions of the data processing apparatuses 400on the sketch map are indicated, for example, by marks 400M such asicons.

In order to designate the data processing apparatus 400 that isrequested to process data on the processing apparatus designation screen160D, the operator preferably performs, for example, an operation oftapping the mark, on the sketch map, indicating the data processingapparatus 400 that is desired to process, for example, the dataprocessing apparatus 400 at the position closest to the terminal device100.

FIG. 16B illustrates a sixth example of the request screen 160Bdisplayed on the display unit 160. In the sixth example of the requestscreen, for example, the processing apparatus image 4001 indicating thedata processing apparatus 400 designated on the processing apparatusdesignation screen 160D illustrated in FIG. 16A is displayed, forexample, on a lower right portion, and, for example, the plural dataimages 960D are further displayed. In the sixth example of the requestscreen 160B, in order to request the data processing apparatus 400 toprocess data, the operator preferably performs, for example, theoperation of dragging the data image 960D indicating the data that isdesired to be processed and then dragging the data image 960D onto theprocessing apparatus image 4001 indicating the data processing apparatus400 that is desired to process.

FIG. 17 illustrates a mode selection screen 160E serving as a screenthat is displayed on the display unit 160 and first displayed after anapplication is activated. The mode selection screen 160E is used forselecting a mode indicating an order and a method in which data that isdesired to be processed and the data processing apparatus 400 that isdesired to process are designated.

A mode selection button 160 h for selecting the camera selection mode inwhich the data processing apparatus 400 that is requested to process isselected before data that is requested to be processed is selected, andthe camera apparatus 170 is used to select the data processing apparatus400 is displayed on the mode selection screen 160E. Further, a modeselection button 160 i for selecting a sketch map mode in which the dataprocessing apparatus 400 that is requested to process is selected beforedata that is requested to be processed is selected, and a sketch map(see FIG. 16A) is used for selection of the processing apparatus isdisplayed on the mode selection screen 160E. Further, a mode selectionbutton 160 j for selecting a data selection mode in which data that isrequested to be processed is selected before the data processingapparatus 400 that is requested to process is selected is displayed onthe mode selection screen 160E.

FIG. 18 illustrates an operation of the data processing system 10. Forexample, when the operator operates the auxiliary operating unit 112 toactivate the application, and thus a series of operations start, in stepS20, the control unit 102 controls the display unit 160 such that themode selection screen 160E is displayed on the display unit 160.

When the operator operates the mode selection button 160 h to select thecamera selection mode in step S22, the process proceeds to the cameraselection mode in step S100. When the operator operates the modeselection button 160 i to select the sketch map selection mode in stepS24, the process proceeds to the sketch map selection mode in step S200.When the operator operates the mode selection button 160 j to select thedata selection mode in step S26, the process proceeds to the dataselection mode in step S300.

On the other hand, for example, when none of the camera selection mode,the sketch map mode, and the data selection mode is selected within apredetermined period of time, the control unit 102 ends the application.

FIG. 19 illustrates the operation of the data processing system 10 inthe camera selection mode. When the control unit 102 confirms that thecamera apparatus 170 has been operated in step S102, the processproceeds to the next step S104. On the other hand, for example, when thecontrol unit does not confirm the operation of the camera apparatus 170within a predetermined period of time, the control unit 102 ends theapplication.

In step S104 of the next step, the control unit 102 acquires theposition information of the terminal device 100 when the cameraapparatus 170 is operated. More specifically, the control unit 102causes the GPS receiving unit 120 to detect the radio waves from the GPSsatellites and acquire the position information of the terminal device100. At this time, preferably, the control unit 102 causes theacceleration sensor 140 to detect the posture of the terminal device 100when the camera apparatus 170 is operated and causes the electroniccompass 150 to detect the direction of the terminal device 100 when thecamera apparatus 170 is operated.

In step S106 of the next step, the control unit 102 requests themanagement apparatus 500 to extract the data processing apparatus 400.More specifically, the control unit 102 controls the wirelesscommunication unit 130 such that the wireless communication unit 130performs communication with the wireless communication apparatus 300,transmits the position information of the terminal device 100 andpreferably the detection results of the posture and the direction of theterminal device 100 to the management apparatus 500 via the network 600,and requests the management apparatus 500 to extract the correspondingdata processing apparatus 400 from the database stored in the storageunit 530 based on the values.

In the next step S108, the control unit 102 acquires the address of thedata processing apparatus 400 extracted in step S106 on the network 600.More specifically, the control unit 102 requests the managementapparatus 500 to extract the address of the data processing apparatus400 extracted in step S106 on the network from the database stored inthe storage unit 530, and further requests the management apparatus 500to transmit the extracted address on the network to the terminal device100.

In the next step S110, the control unit 102 causes the display unit 160to display the processing apparatus display screen 160A. At this time,when the data processing apparatus 400 extracted in step S106 is shownin the photograph captured through the camera apparatus 170 in stepS102, the photograph captured through the camera apparatus 170 is usedas the processing apparatus image 4001, and the image of the dataprocessing apparatus 400 in the photograph is used as the processingapparatus image 4001 (see the first example of the processing apparatusdisplay screen 160A illustrated in FIG. 7B).

On the other hand, for example, when the data processing apparatus 400extracted in step S106 is arranged at the position hidden by anobstacle, and thus the data processing apparatus 400 is not shown in thephotograph captured in step S102, the control unit 102 uses the image ofthe data processing apparatus 400 that is photographed in advance andstored in the storage unit 530 of the management apparatus 500, the iconindicating the data processing apparatus 400, or the like as theprocessing apparatus image 4001 (see the second example of theprocessing apparatus display screen 160A illustrated in FIG. 15A).

In step S112 of the next step, the control unit 102 causes the displayunit 160 to display the request screen 160B. In other words, an imagedisplayed so that at least one data image 960D is superposed on theprocessing apparatus display screen 160A displayed in step S110 isdisplayed on the display unit 160 (for example, see the first example ofthe request screen 160B illustrated in FIG. 8A and the fourth example ofthe request screen 160B illustrated in FIG. 15C).

When the operator designates the data image 960D indicating data to beprocessed, and designates the processing apparatus image 4001 indicatingthe data processing apparatus that is requested to be processed in stepS114 of the next step, in step S116, the control unit 102 transfers thedata indicated by the selected data image 960D to the data processingapparatus 400 through the wireless communication apparatus 300 and thenetwork 600.

For example, when the data image 960D and the processing apparatus image4001 are not selected within a predetermined period of time in stepS114, the control unit 102 ends the application.

FIG. 20 describes the operation of the data processing system 10 in thesketch map selection mode. When the sketch map selection mode isselected, in step S202, the control unit 102 acquires the positioninformation of the terminal device 100 at a point in time at which thesketch map selection mode is selected. More specifically, the controlunit 102 causes the GPS receiving unit 120 to detect the radio wavesfrom the GPS satellites and acquire the position information of theterminal device 100.

In step S204 of the next step, the control unit 102 causes the displayunit 160 to display the processing apparatus designation screen 160Dindicating the sketch map of the area around the position of theterminal device 100, the position of the terminal device 100 in thesketch map, and the position of the data processing apparatus 400 in thesketch map based on the position information of the terminal device 100acquired in step S202 (see the processing apparatus designation screen160D illustrated in FIG. 16A). More specifically, the control unit 102transmits the position information of the terminal device 100 acquiredin step S204 to the management apparatus 500, requests the managementapparatus 500 to generate the processing apparatus designation screen160D and transfer the generated processing apparatus designation screen160D to the terminal device 100, and causes the display unit 160 todisplay the processing apparatus designation screen 160D transmittedfrom the management apparatus 500.

At this time, the control unit 502 of the management apparatus 500generates the processing apparatus designation screen 160D based on theposition data of the terminal device 100 transmitted from the terminaldevice 100, the position data of the data processing apparatus 400stored in the storage unit 530, and the sketch map stored in the storageunit 530.

When the operator designates the mark 400M indicating the dataprocessing apparatus 400 that is requested to process in step S206 ofthe next step, in step S208 of the next step, the control unit 102acquires the address of the data processing apparatus 400 whose mark400M is selected in step S206 on the network 600. More specifically, thecontrol unit 102 requests the management apparatus 500 to transmit theaddress of the data processing apparatus 400 whose mark 400M is selectedon the network 600.

In step S212 of the next step, the control unit 102 causes the displayunit 160 to display the request screen 160B (see the fifth example ofthe request screen 160B illustrated in FIG. 16B). More specifically, instep S210, the control unit 102 causes the management apparatus 500transfer the processing apparatus image 4001 of the data processingapparatus 400 whose mark 400M is selected, causes the display unit 160to display the transferred processing apparatus image 4001, and furthercauses the display unit 160 to display the data image 960D.

When the operator designates the data image 960D indicating data to beprocessed in step S214 of the next step, in step S216, the control unit102 controls the wireless communication unit 130 such that the dataindicated by the selected data image 960D is transferred to the dataprocessing apparatus 400 indicated by the mark 400M selected in stepS206 through the wireless communication apparatus 300 and the network600.

For example, when the processing apparatus image 4001 is not selectedwithin a predetermined period of time in step S214, the control unit 102ends the application.

FIG. 21 illustrates the operation of the data processing system 10 inthe data selection mode. When the data selection mode is selected, instep S302, the control unit 102 causes the display unit to display thedata designation screen 160C (see FIGS. 12A to 12D).

When the operator designates the data image 960D in the next step S304,in step S306 of the next step, the control unit 102 causes the displayunit 160 to display the data display screen 160F, and the data displayscreen 160F is a screen on which the data image 960D selected in stepS304 is displayed, for example, on the upper left portion of the displayunit 160.

When the camera apparatus 170 is operated in step S308 of the next step,in step S310, the control unit 102 acquires the position information ofthe terminal device 100 based on the radio waves that are received fromthe GPS satellites 990 through the GPS receiving unit 120. On the otherhand, when the camera apparatus 170 is not operated during apredetermined period of time in step S308, the control unit 102 ends theapplication.

In step S312 of the next step, the control unit 102 requests themanagement apparatus 500 to extract the data processing apparatus 400.More specifically, the control unit 102 controls the wirelesscommunication unit 130 such that the wireless communication unit 130performs communication with the wireless communication apparatus 300,transmits the position information of the terminal device 100 andpreferably the detection results of the posture and the direction of theterminal device 100 to the management apparatus 500 via the network 600,and requests the management apparatus 500 to extract the correspondingdata processing apparatus 400 from the database stored in the storageunit 530 based on the values.

In step S314 of the next step, the control unit 102 acquires the addressof the data processing apparatus 400 extracted in step S312 on thenetwork 600. More specifically, the control unit 102 requests themanagement apparatus 500 to extract the address of the data processingapparatus 400 extracted in step S106 on the network 600 from thedatabase stored in the storage unit 530, and requests the managementapparatus 500 to transmit the extracted address on the network to theterminal device 100.

In step S316 of the next step, the control unit 102 causes the displayunit 160 to display the request screen 160B. In other words, the imagedisplayed so that the processing apparatus image 4001 indicating thedata processing apparatus 400 in which the address on the network isacquired in step S314 is superposed on the data display screen 160Fdisplayed in step S306 is displayed on the display unit 160 (see thethird example of the request screen 160B illustrated in FIG. 13B).

When a data processing instruction is given in step S318 of the nextstep, in step S320, the control unit 102 controls the wirelesscommunication unit 130 such that the wireless communication unit 130transfers the data indicated by the selected data image 960D to the dataprocessing apparatus 400 through the wireless communication apparatus300 and the network 600.

For example, when the processing apparatus image 4001 is not selectedwithin a predetermined period of time in step S318, that is, when thedata processing instruction is given, the control unit 102 ends theapplication.

Second Exemplary Embodiment

Next, a second exemplary embodiment of the present invention will bedescribed. FIG. 22 illustrates a terminal device 100 according to thesecond exemplary embodiment of the present invention. In the firstexemplary embodiment, the smart phone is used as the terminal device 100(see FIG. 3). On the other hand, in the second exemplary embodiment, aglasses type of wearable computer is used as the terminal device 100.For example, the wearable computer forms an image as a 3D hologram, anddisplays the image formed as the 3D hologram is superimposed on a realimage viewed by the operator.

The terminal device 100 includes a housing portion 104, a frame portion106, and a lens portion 108 as illustrated in FIG. 22. The lens portion108 is mounted in the frame portion 106, similarly to common glasses.The housing portion 104 is mounted in the frame portion 106 such thatthe field of view of the operator by the lens portion 108 is notblocked.

The camera apparatus 170 is mounted in the housing portion 104. Thecontrol unit 102, the GPS receiving unit 120, the wireless communicationunit 130, the acceleration sensor 140, the electronic compass 150, andthe memory 180 are installed in the housing portion 104. The GPSreceiving unit 120, the wireless communication unit 130, theacceleration sensor 140, the electronic compass 150, and the memory 180have the same configurations as those in the first exemplary embodiment,and thus a description thereof is omitted.

The terminal device 100 does not include the operating unit 110 such asthe touch panel with which the terminal device 100 according to thefirst exemplary embodiment is equipped. In the terminal device 100, forexample, an operation is performed by detecting motion of the hand ofthe operator and the position information through the camera apparatus170 and processing the detected motion of the hand and the positioninformation through the control unit 102.

In the terminal device 100 according to the first exemplary embodiment,the LCD panel is used as the display unit 160. On the other hand, in thesecond exemplary embodiment, a half mirror member 162 is used as thedisplay unit 160. The display unit 160 includes a light emitting element164 and an optical system 166 along with the half mirror member 162. Thelight emitting element 164 and the optical system 166 are mounted in thehousing portion 104 so that light emitted from the light emittingelement 164 forms an image on the half mirror member 162 through theoptical system 166.

The half mirror member 162 is formed of a transparent material such asresin or glass. For example, An LCD element or an electroluminescenceelement may be used as the light emitting element 164. For example, oneof optical elements such as a lens and a prism or a combination thereofmay be used as the optical system 166.

When no image is displayed on the half mirror member 162, the displayunit 160 enters a state in which the half mirror member 162 becomestransparent and transmits light. Thus, the field of view of the operatoris not blocked in the state in which no image is displayed on the halfmirror member 162.

FIG. 23 illustrates a request screen 162B used for requesting the dataprocessing apparatus 400 to process data. As illustrated in FIG. 23,data images 960D indicating data that is requested to be processed aredisplayed on the request screen 162B so that the operator can view thedata images 960D at the same time as the real image of the dataprocessing apparatus 400. In FIG. 23, the data processing apparatus 400and the clock 922 are the real images which the operator is viewingthrough the half mirror member 162 and the lens portion 108. On theother hand, the data image 960D is an image that is formed by theoptical system 166 using the light emitted from the light emittingelement 164 and displayed.

In order for the operator to request the data processing apparatus 400to process data on the request screen 160B, it is desirable to move thedata image 960D indicating the data that is desired to be processed ontothe real image of the data processing apparatus 400 through a gesture.The gesture of the operator is recognized by detecting, for example,motion of the hand and the position information of the operator throughthe camera apparatus 170 and processing the detected motion of the handand the position information through the control unit 102. Instead ofdetecting, for example, the motion of the hand and the positioninformation of the operator through the camera apparatus 170, theoperator may wear the wearable terminal 810 (see FIGS. 9A and 9B), andthe motion of the hand and the position of the operator may be detectedby detecting the wearable terminal.

In the terminal device 100 according to the second exemplary embodiment,it is desirable that the data image 960D be not displayed while theoperator is moving. Thus, although the operator is moving, it ispossible to prevent 3D sickness of the operator that may occur as thedata image 960D is displayed.

In the terminal device 100 according to the second exemplary embodiment,it is desirable to cause the data image 960D to be displayed in two orless colors. Thus, it is possible to make it easier for the operator todistinguish, for example, the real image of the data processingapparatus 400 or the clock 922 from the data image 960D than when thedata image 960D is displayed in three or more colors.

In the terminal device 100 according to the second exemplary embodiment,it is desirable that the data image 960D be permeably displayed. Thus,it is possible to suppress an adverse effect that when an opaque dataimage 960D is displayed, the real image is blocked by the data image960D, and, for example, the real image of the data processing apparatus400 or the clock 922 is difficult to view.

In the data processing system 10 according to the second exemplaryembodiment, a configuration and an operation excluding theabove-described configuration and the moving image are the same as thoseof the data processing system 10 according to the first exemplaryembodiment. Thus, a description of the configuration and the operationwhich are the same as in the first exemplary embodiment is omitted.

Third Exemplary Embodiment

Next, a third exemplary embodiment of the present invention will bedescribed. FIG. 24 illustrates a data processing system 10 according tothe third exemplary embodiment of the present invention. In the firstexemplary embodiment, the data processing system 10 includes a terminaldevice 100, a wireless communication apparatus 300, for example, threedata processing apparatuses 400A, 400B, and 400C, and a managementapparatus 500 which are connected to a network 600. In addition to thesecomponents, the data processing system 10 according to the thirdexemplary embodiment further includes a data storage apparatus 840 and agesture delivery apparatus 860, and the data storage apparatus 840 andthe gesture delivery apparatus 860 are connected to the network 600.

In the first exemplary embodiment, the example in which one terminaldevice 100 is connected to the network 600 has been described, but thethird exemplary embodiment will be described in connection with anexample in which three terminal devices 100A, 100B, and 100C areconnected to the network 600. Here, the terminal devices 100A, 100B, and100C are assumed to differ in a used platform (an operating system(OS)). The third exemplary embodiment will be described in connectionwith an example in which one operator wearing the wearable terminal 810(see FIGS. 9A and 9B) operates the terminal device 100A using a gesture,and the terminal devices 100B and 100C are operated by other operators.

The data storage apparatus 840 is a so-called document server and storesdata that is requested to be processed by the data processing apparatus400.

For example, the gesture delivery apparatus 860 delivers the gesture ofthe operator operating one terminal device, for example, the terminaldevice 100A to the other terminal devices, for example, the terminaldevices 100B and 100C. More specifically, the gesture delivery apparatusincludes a detecting apparatus 862 that detects motion and positioninformation of the wearable terminal 810 worn on the operator, anddelivers the motion and the position of the wearable terminal 810detected through the detecting apparatus 862 to the other terminaldevices 100B and 100C via the network 600.

In the data processing system 10 according to the third exemplaryembodiment, for example, when an operation of designating the processingapparatus image 4001 indicating the data processing apparatus 400 thatis requested to process data or selecting the data image 960D that isrequested to be processed is performed in the terminal device 100Athrough the gesture, the gesture is delivered to the terminal devices100B and 100C through the gesture delivery apparatus 860, and thus theoperation performed on the terminal device 100A can be displayed on thedisplay units 160 with which the terminal devices 100B and 100C areequipped. In the data processing system 10 according to the thirdexemplary embodiment described above, the other operators can confirmthe operation performed by one operator using one terminal device 100using the other terminal devices 100.

As described above, the present invention can be applied to a terminaldevice, a data processing system, and a program.

The foregoing description of the exemplary embodiments of the presentinvention has been provided for the purposes of illustration anddescription. It is not intended to be exhaustive or to limit theinvention to the precise forms disclosed. Obviously, many modificationsand variations will be apparent to practitioners skilled in the art. Theembodiments were chosen and described in order to best explain theprinciples of the invention and its practical applications, therebyenabling others skilled in the art to understand the invention forvarious embodiments and with the various modifications as are suited tothe particular use contemplated. It is intended that the scope of theinvention be defined by the following claims and their equivalents.

What is claimed is:
 1. A terminal device, comprising: a photographingapparatus that captures a photograph in a specific direction in which adata processing apparatus is installed; a display unit that displays arequest screen used for designating a data processing apparatus toprocess data, the request screen configured to display the photographcaptured by the photographing apparatus, at least one of data imagesindicating data that is requested to be processed, and, when an image ofthe data processing apparatus arranged in the specific direction is notcaptured in the photograph due to an obstacle positioned between thephotographing apparatus and the data processing apparatus, a processingapparatus image indicating the data processing apparatus acquired over anetwork; and a control unit configured to: acquire position informationof the data processing apparatus; acquire the processing apparatus imageindicating the data processing apparatus over the network; acquire anaddress of the data processing apparatus on the network; and in responseto an operation of an operator of designating the processing apparatusimage and one of the data images on the request screen, transmit arequest for processing the data indicated by the designated data imageto the data processing apparatus indicated by the designated processingapparatus image.
 2. The terminal device according to claim 1, wherein:the display unit further displays a data designation screen used fordesignating the data that is requested to be processed, and the pluraldata images are displayed on the data designation screen such that theplural data images are arranged to have a depth in a direction crossingthe data designation screen.
 3. The terminal device according to claim1, wherein the data image and the processing apparatus image aredesignated on the request screen by the operator causing one of the dataimages to overlap one of the processing apparatus images.
 4. Theterminal device according to claim 1, wherein the display unit does notdisplay the data image while the operator is moving.
 5. The terminaldevice according to claim 1, wherein the display unit displays the dataimage in two or less colors.
 6. The terminal device according to claim1, wherein the display unit displays the data image permeably.
 7. Anon-transitory computer readable medium storing a program causing acomputer to execute: capturing a photograph by a photographing apparatusin a specific direction in which a data processing apparatus isinstalled; displaying a request screen used for designating a dataprocessing apparatus to process data, the request screen configured todisplay the photograph captured by the photographing apparatus, at leastone of data images indicating data that is requested to be processed,and, when an image of the data processing apparatus arranged in thespecific direction is not captured in the photograph due to an obstaclepositioned between the photographing apparatus and the data processingapparatus, a processing apparatus image indicating the data processingapparatus acquired over a network; acquiring position information of thedata processing apparatus; acquiring the processing apparatus imageindicating the data processing apparatus over the network; acquiring anaddress of the data processing apparatus on the network; and in responseto an operation of an operator of designating the processing apparatusimage and one of the data images on the request screen, transmitting arequest for processing the data indicated by the designated data imageto the data processing apparatus indicated by the designated processingapparatus image.